High-speed event imaging device

ABSTRACT

A high speed video camera capable of taking video of a high speed phenomena by an image sensor composed of a matrix of MOS type light sensor circuits each representing a unit pixel and having a wide dynamic range with no occurrence of afterglow, recording the video by storing image data output from the image sensor, executing compensation of outputs of the image sensor based on the image data read from the memory at a normal processing speed and displaying the video based on the compensated video data on a display device. Accordingly, with this camera, it is possible for the user to take and record video of a high speed phenomenon at a very high speed while executing compensation for variations in outputs of the image sensor at a normal processing speed based on the video data read from the memory.

BACKGROUND OF THE INVENTION

The present invention relates to a high-speed video camera capable oftaking images of high speed phenomena, storing data of the images on amemory and reproducing the images based on the image data stored on thememory.

If a conventional video camera is used for taking video of a high-speedphenomenon that may unexpectedly occur and does not allow the user topreset optimal photographing conditions, it cannot capture the subjectin time with its automatic diaphragm, resulting in viewing fuzzy videowith considerable afterglow of pixels therein on a display screen.

In recent years, there has been developed an image sensor comprising amatrix of MOS type light sensor circuits each representing a unit pixel,which has a wide dynamic range suitable for capturing high speedphenomena to visualize with least occurrence of afterglow of pixels.

Japanese Laid-Open Patent Publication No. 2000-329616 discloses an imagesensor using a number of light sensor circuits each representing a unitpixel, which circuit comprises, as shown in FIG. 1, a photo-diode PDoperating as a photoelectric converting element for producing a sensorcurrent proportional to the quantity of incident light Ls fallingthereon, a transistor Q1 having a logarithmic output characteristic in aweak inverse state for converting the sensor current produced in thephotodiode into a voltage signal Vpd, a transistor Q2 for amplifying thevoltage signal Vpd and a transistor Q3 for outputting a sensor signal inaccordance with a timing pulse Vs and which circuit is characterized byits wide dynamic range, thereby achieving the high sensitivity ofdetecting a light signal. In addition, the light sensor circuit isprovided with a means for changing a drain voltage VD of the transistorQ1 to a value lower than a normal value for a specified period to removea charge accumulated in a parasitic capacitor C of the photodiode PD toinitialize the circuit. The light sensor circuit can thus obtain avoltage signal Vpd corresponding to the quantity of incident light Lseven if the sensor current absurdly changed, thereby eliminating thepossibility of occurrence of afterglow of the pixel even at a smallquantity of incident light.

However, the conventional image sensor using the above-described lightsensor circuits cannot be free from structure-derived variations inoutput characteristics as well as variations in temperaturecharacteristics of pixel signals. In other words, both kinds of thevariations shall be compensated for, otherwise, dark-current andbright-current image data cannot be obtained when taking images. Thecompensation of the image sensor for variations in its outputcharacteristics may be carried out by performing operations referring toa table containing compensating values predetermined based on themeasured variations of every sensor circuit. In the case when theconventional image sensor is used for taking an image from a high speedphenomenon by temporally storing obtained image data on a high speedmemory and displaying the data on a display screen, it is needed topromptly compensate the image data for variations in outputcharacteristics of respective pixels before storing the image data onthe high speed memory. However, it is very difficult to provide theimage sensor with a compensating circuit capable of compensating pixeloutputs of the image sensor in time at a speed high enough to follow thehigh speed phenomenon.

Thus, when a high speed subject is captured, fast stored and displayedby the conventional image sensor using a matrix of MOS type light sensorcircuits (pixels) having a wide dynamic range with least occurrence ofafterglow of pixels, there still arises a problem that it is almostimpossible to compensate the image sensor for variations in outputcharacteristics of respective pixels before storing data of the capturedimage on a memory by following the high speed phenomenon.

Furthermore, when displaying an image taken from the high speedphenomenon being captured by the image sensor in order to monitor thestate of taking video thereof, there may be such a problem that theframe rate of the image sensor exceeds the reproducing speed of thedisplay device which in this case cannot display the image.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a high speed videocamera capable of taking video of high speed phenomena by using ahigh-speed image sensor composed of a number of MOS type light sensorcircuits, each representing a unit pixel and possessing a wide dynamicrange not to cause afterglow of pixels, by which image data istemporally stored on a high speed memory and displaying a video signalon a display screen, wherein, in order to easily realize compensationfor variations in output characteristics of the image sensor, image dataoutput from the image sensor is directly stored on the high speed memoryand then compensated for variations in output characteristics ofrespective pixels based on the image data stored and readout from thememory, and an image sensor signal is reproduced based on thecompensated image data and displayed on the display screen.

Another object of the present invention is to provide a high speed videocamera provided with a separate memory for storing image data on acaptured high speed phenomenon at a framing rate adapted to that of amonitor display device, which is thus capable of reading the image datafrom the memory, compensating for variations in output characteristicsof respective light sensor outputs of the image sensor and presenting animage signal of the high speed phenomenon based on the compensated imagedata on a monitoring screen of the display device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an electric circuit diagram of a light sensor circuit for onepixel, which is used as a unit component of an image sensor of a highspeed video camera according to the present invention.

FIG. 2 is a time chart of signals to be generated at respective portionsof the light sensor circuit of FIG. 1.

FIG. 3 shows a characteristic of a pixel output signal versus a lightsensor current of FIG. 1.

FIG. 4 is a basic block diagram of an image sensor.

FIG. 5 is a time chart of signals to be generated at respective portionsof the image sensor of FIG. 4.

FIG. 6 is a block diagram showing an exemplary construction of an outputcompensating device for compensating for variations in outputcharacteristics of an image sensor.

FIG. 7 is a flowchart depicting the operation of the output compensatingdevice of FIG. 6.

FIG. 8 shows exemplary variations in output characteristics of pixelsignals from the image sensor, which variations were derived from thestructure of respective light sensor circuits of the image sensor.

FIG. 9 shows output characteristics of pixel signals, which wereobtained by offset compensation of the signals having the outputcharacteristics shown in FIG. 8.

FIG. 10 shows output characteristics of pixel signals, which wereobtained by offset and gain compensations of the signals having theoutput characteristics shown in FIG. 8.

FIG. 11 is a block diagram showing another exemplary construction of anoutput compensating device for compensating for variations in outputcharacteristics of pixel signals from an image sensor, which variationswere derived from variations in temperature characteristics ofrespective light sensor circuits of the image sensor.

FIG. 12 is a flowchart depicting the operation of the outputcompensating device of FIG. 11.

FIG. 13 shows an example of variations in output characteristics ofpixel signals from an image sensor, which variations were derived fromvariations in temperature characteristics of respective light sensorcircuits of the image sensor.

FIG. 14 shows output characteristics of pixel signals, which wereobtained by offset compensation of temperature characteristics of thesignals having the output characteristics shown in FIG. 13.

FIG. 15 shows output characteristics of pixel signals, which wereobtained by offset compensation and gain compensation of temperaturecharacteristics of the signals having the output characteristics shownin FIG. 13.

FIG. 16 shows two output characteristics of pixel signals, one of whichis obtained with no effect of a temperature offset and other is obtainedwith an effect of a temperature offset.

FIG. 17 is a flowchart depicting the operation of the outputcompensating device for compensating for variations in outputcharacteristics of an image sensor, which variations were derived fromvariations in output characteristics and temperature characteristics ofrespective light sensor circuits of the image sensor.

FIG. 18 is an another exemplary flowchart depicting the operation of theoutput compensating device for compensating for variations in outputcharacteristics of an image sensor, which variations were derived fromvariations in output characteristics and temperature characteristics ofrespective light sensor circuits of the image sensor.

FIG. 19 is another exemplary flowchart depicting the operation of anoutput compensating circuit of an output compensating device shown inFIG. 6.

FIG. 20 shows another example of variations in output characteristics ofpixel signals of an image sensor, which variations were derived from thestructure of respective pixel circuits of an image sensor.

FIG. 21 shows output characteristics of pixel signals, which wereobtained by offset compensation of the signals having the outputcharacteristics shown in FIG. 20.

FIG. 22 shows output characteristics of pixel signals, which wereobtained by offset compensation and gain compensation of the signalshaving the output characteristics shown in FIG. 20.

FIG. 23 is another exemplary flowchart depicting the operation of theoutput compensation device of FIG. 11.

FIG. 24 shows another example of variations in output characteristics ofpixel signals from an image sensor, which variations may be caused fromvariations in temperature characteristic of respective light sensorcircuits.

FIG. 25 shows output pixel signal characteristics obtained by offsetcompensation of temperature characteristics of the signals having theoutput characteristics shown in FIG. 24.

FIG. 26 shows output characteristics of pixel signals obtained by offsetcompensation and gain compensation of temperature characteristics of thesignals having the output characteristics shown in FIG. 24.

FIG. 27 is another exemplary flowchart depicting the operation of anoutput compensation device for compensating for variations in outputcharacteristics and temperature characteristics of respective lightsensor circuits of an image sensor.

FIG. 28 is a further exemplary flowchart depicting the operation of anoutput compensation device for compensating for variations in outputcharacteristics and temperature characteristics of respective lightsensor circuits of an image sensor.

FIG. 29 is a general block construction diagram of a high speed videocamera.

FIG. 30 is a block construction diagram of a high speed video cameraaccording to another embodiment of the present invention.

FIG. 31 is a block construction diagram of a high speed video cameraaccording to still another embodiment of the present invention.

FIG. 32 is a block construction diagram of a high speed video cameraaccording to a further embodiment of the present invention.

PREFERRED EMBODIMENTS OF THE PRESENT INVENTION

An image sensor used for a high speed video camera according to thepresent invention uses as a unit pixel a light sensor circuitillustrated in FIG. 1.

FIG. 2 shows a time chart of signals produced at various portions of thelight sensor circuit. In FIG. 2, t1 is an initializing timing pulse andt2 is a light-signal detection timing pulse. T designates a period foraccumulating a charge in a parasitic capacitor C of the photodiode PD.

The light sensor circuit, as illustrated in FIG. 3, has a logarithmicoutput characteristic with a sufficient sensor current produced in aphotodiode PD, i.e., a sufficient quantity of light falling thereon, buthas a non-logarithmic (almost linear) output characteristic with a smallcurrent in the photodiode due to a response lag caused in charging aparasitic capacitor of the photodiode PD. In FIG. 3, WA designates anon-logarithmic response region and WB designates a logarithmic responseregion.

FIG. 4 shows an exemplary construction of an image sensor consisting ofa number of the light sensor circuits (FIG. 1) arranged to form a matrixof pixels, wherein sensor signals from respective pixels are read byscanning in a time series and the pixels can be initialized at timingadapted to the readout-scanning of respective sensor signals.

The image sensor is composed of, for example, 4×4 pixels D11˜D44arranged in a matrix of pixel circuits, in which pixel lines areselected one by one with respective selecting signals LS1˜LS4successively output from a pixel line selecting circuit 1 and pixels ineach selected pixel line are readout one by one as respective sensorsignals in such a manner that selecting signals DS1˜DS4 successivelyoutput from a pixel selecting circuit 2 turn on corresponding switchesSW1˜SW4 (in a group 3) to read respective pixel signals Vo in a timeseries. In FIG. 4, numeral 4 designates a power source for gate voltageVG of the transistor Q1 and numeral 6 designates a power source for adrain voltage VD of the transistor Q1.

The image sensor is provided with a voltage switching-over circuit 5 bywhich a drain voltage VD of each transistor Q1 for each pixel is changedfrom a normal high-level H to an initializing lower level L and reverseby the effect of specified timing pulses when selecting each line ofpixels.

FIG. 5 shows a time chart of signals generated at respective portions ofthe above-described image sensor during the operation of the imagesensor.

The above-described construction of the image sensor is the same as thatof the conventional image sensor described before.

In the thus constructed image sensor, a sensor current corresponding toa quantity of light falling on each sensor circuit is converted into avoltage by a transistor Q1 by using its sub-threshold regioncharacteristic. However, respective sensor circuits have respectivesub-threshold values which may be different from each other andtherefore may cause variations in pixel output characteristics of theimage sensor. In addition, the amplifying transistors Q2 provided foramplifying voltage signals received with high impedance fromcorresponding transistors Q1 to output amplified voltage signals may bedifferent in their characteristics, which may also cause variations inpixel output characteristics of the image sensor.

Furthermore, the image sensor cannot be free from variations intemperature characteristics of light sensor circuits.

FIG. 6 shows a basic construction of an output compensating system forconducting offset and gain compensations for variations in outputcharacteristics of pixel signals, which variations were derived from thestructures of corresponding light sensor circuits.

This system comprises an image sensor 7, an ECU 8 for controlling theoperation for reading pixel signals in a time series, an A-D converter 9for converting pixel signals Vo outputted in a time series from theimage sensor 7 into corresponding digital signals, a memory 10 forstoring offset compensation values OFS predetermined for outputcharacteristics of pixels (light sensor circuits) and multipliers MLTfor gain compensation, both of which can be selected in accordance withan address signal ADDRESS (X, Y) of a pixel to be processed, and anoutput compensating circuit 11 for performing arithmetic operationsnecessary for the offset and gain compensations of the digitized pixelsignals by using corresponding offset compensation values OFS andmultipliers MLT read from the memory 10.

FIG. 8 shows an example of different output characteristics of threepixel signals A, B and C which differences were caused from thestructure-derived variations in the output characteristics ofcorresponding light sensor circuits. In the shown example, a sensorcurrent value Im corresponding to a threshold H of a pixel outputrepresents a point at which characteristics of pixel signals A, B and Cchange from a non-logarithmic response region to a logarithmic responseregion. Io designates a dark current in a sensor when it is notilluminated.

According to the present invention, the compensation of outputs of theimage sensor is normally conducted when the output characteristics ofrespective pixel signals are substantially the same in shape in thenon-logarithmic response region WA but they are different by gradientfrom each other in the logarithmic response region WB. Parameters foreach pixel signal are information about the point at which itscharacteristic changes from the non-logarithmic response region WA tothe logarithmic response region WB and a pixel output appearing at adark sensor current.

FIG. 7 illustrates the operation of the output compensating circuit 11.

In the memory 10, there is a table of offset compensation values OFS forcorrecting outputs of respective pixel signals so as to attain a value Hat a sensor current of Im. In an offset compensating portion 111, thedigitized pixel signals DS are corrected by arithmetic operations(addition, subtraction) using corresponding offset compensation valuesOFS. As the result of the offset compensation, three pixel signals (A, Band C) have the same characteristic in the non-logarithmic responseregion WA as shown in FIG. 9.

In a gain compensating portion 112, the gain compensation of outputcharacteristics of three pixel signals in the logarithmic responseregion WB above the threshold value H is conducted by arithmeticoperations (multiplication) using corresponding multipliers based on theoffset-compensated signals DS1.

In practice, the offset-compensated pixel signal DS1 is checked whetherit is greater than the threshold value H and, if so (i.e., the signal isin the logarithmic region WB), it is further subjected to gaincompensation by the following arithmetic operations using a specifiedmultiplier MLT read from the memory 10.Output←H+(Pixel Signal DS1−H)×MultiplierA result signal is output as an output-compensated pixel signal DS2.

As the result of the above-described gain compensation, three pixelsignals A, B and C have the same characteristics in the logarithmicresponse region WB as shown in FIG. 10. In this instance, theoffset-compensated pixel signal DS1 being smaller than the thresholdvalue (i.e., in the non-logarithmic response region WA) is directlyoutput as an output-compensated digital pixel signal DS2.

FIG. 11 shows a basic construction of a system for compensating pixelsignals for offset values and gain levels, which differences wereresulted from variations in temperature characteristics of correspondingpixel circuits.

The system comprises an image sensor 7 incorporating a temperaturesensor 12, an ECU 8 for performing the control of reading pixel signalsVo from respective pixel circuits in a time series and temperaturedetection signals TS from the temperature sensor 12 at a specifiedtiming, an A-D converter 9 for converting pixel signals Vo output in atime series from the image sensor 7 into corresponding digital signals,an A-D converter 13 for converting the temperature detection signals TSread from the temperature sensor 12 into corresponding digital signals,a memory 14 for storing offset compensation values T-OFS predeterminedfor temperature characteristics of respective pixel circuits andmultipliers T-MLT for gain compensation, which values can be selected inaccordance with the digitized temperature detection signal DTS, and anoutput compensating circuit 15 for performing arithmetic operationsnecessary for the offset compensation and gain compensation of thedigitized pixel signal using a corresponding offset compensation valueOFS and multiplier MLT read from the memory 14.

FIG. 13 shows exemplary variations in output characteristics ofrespective pixel signals TA, TB and TC in accordance with temperatures.In the shown example, a sensor current value Itm corresponding to apixel output threshold TH represents a point at which pixel signals TA,TB and TC, corresponding to temperatures, change from a non-logarithmicresponse region WA to a logarithmic response region WB. Io designates adark current in a sensor when it is not illuminated.

According to the present invention, the compensation of outputs of theimage sensor is normally conducted when the output characteristics ofrespective pixel signals have the substantially same shape in thenon-logarithmic response region WA but they are different by gradientfrom each other in the logarithmic response region WB. Parameters forthe pixel signals are information about the point at whichcharacteristics of pixel signals corresponding to the temperatures (TA,TB and TC) change from the non-logarithmic response region WA to thelogarithmic response region WB and the pixel output with a dark sensorcurrent.

FIG. 12 illustrates the operation of the output compensating circuit 15.

In the memory 14, an offset compensation value T-OFS for obtaining thepixel output of the threshold value TH at the sensor current of Itm. Inan offset compensating portion 151, the digitized pixel signals DS areprocessed by arithmetic operations (addition, subtraction) using theoffset compensation value T-OFS. As the result of the offsetcompensation, three pixel signals TA, TB and TC, corresponding to thetemperatures, have the same characteristics in the non-logarithmicresponse region WA as shown in FIG. 14.

In a gain compensating portion 152, the gain compensation ofcharacteristics of the pixel signals in the logarithmic response regionWB above the threshold value TH is conducted by arithmetic operations(multiplication) using a corresponding multiplier and the offsetcompensated signal DS1′.

In practice, the offset-compensated pixel signal DS1′ is checked whetherit is greater than the threshold value TH and, if so, it is furthersubjected to gain compensation by conducting the following arithmeticoperations using a specified multiplier T-MLT selected from the memory14.Output←TH+(Pixel Signal DS1′−TH)×MultiplierA result signal is outputted as an output-compensated digital pixelsignal DS2′.

As the result of the gain compensation, three pixel signals TA, TB andTC, with corresponding temperatures, have the same characteristic in thelogarithmic response region WB as shown in FIG. 15. In this instance,the offset-compensated pixel signal DS1′ being smaller than thethreshold value TH is directly outputted as an output-compensateddigital pixel signal DS2′.

The output compensating device for the image sensor according to thepresent invention conducts the offset and gain compensations forvariations in the output characteristics of pixel signals, whichvariations were resulted from the structures of pixel circuits, and theoffset and gain compensations for variations in the temperaturecharacteristics of the same pixel signals so that the pixel signals ofthe image sensor are free from the effects of both kinds ofcharacteristic variations of the pixel circuits can be obtained.

If a pixel signal affected by both kinds of the characteristicvariations was compensated first for variations in outputcharacteristics by the offset and gain compensations according to themethod shown in FIG. 7 and then compensated for variations intemperature characteristics by the offset and gain compensationsaccording to the method shown in FIG. 12, the compensated pixel signalmay not completely be corrected because the above compensation was madewithout compensation for the variation caused by a change intemperature. In other words, conducting the offset and gaincompensations of the pixel signal for the structure-derived variation inits output characteristic in respective regions with a boundary of athreshold level H, no problem may arise as far as the pixel signal isnot subjected to the effect of a change in temperature as shown by a dotline in FIG. 16. However, if the signal was subjected to a temperatureoffset as shown by a solid line in FIG. 16, the compensation regionboundary point is shifted from “a” to “a′” since the level H is fixed.As the result, the compensation was made in the different irregular way.In the shown case, the output characteristic of the pixel signal isshifted downward, resulting in shifting the level H above the boundarybetween the non-logarithmic response region WA and the logarithmicresponse region WB.

The above-described problem can be solved by the present invention insuch a manner that the level H is aligned with the boundary between thenon-logarithmic response region WA and the logarithmic response regionWB by conducting the offset compensation for variations in temperaturecharacteristics of respective pixel signals before conducting the offsetand gain compensations for structure-derived variations in outputcharacteristics of the pixel signals.

The same kind of problem may arise when conducting the offset and gaincompensations for variations in temperature characteristics and then theoffset and gain compensations for structure-derived variations in outputcharacteristics of respective pixel signals. In this case, the level THis aligned with the boundary between the non-logarithmic response regionWA and the logarithmic response region WB by conducting the offsetcompensation for the structure-derived variation in the outputcharacteristics of each pixel signal before conducting the offsetcompensation for the variation in temperature characteristic of eachpixel signal.

FIG. 17 shows a procedure for conducting the offset compensation of eachpixel signal for a variation in temperature characteristic, the offsetand gain compensation for a structure-derived variation in outputcharacteristic and then the offset and gain compensation for a variationin temperature characteristic.

In FIG. 17, a block 16 is similar to the processing block 112 shown inFIG. 7 for conducting the offset and gain compensation for a variationin the output characteristic of each pixel signal, and a block 17 issimilar to the processing block 14 shown in FIG. 12 for conducting theoffset and gain compensation for a variation in the temperaturecharacteristic of each pixel circuit. In this case, a pixel signal DSfrom the image sensor 7 is digitized and transferred to an offsetcompensation block 151 on the side of the temperature characteristiccompensation block 17, whereby the pixel signal DS is subjected to theoffset compensation for its temperature characteristic variation toattain a correct level H necessary for compensating for the outputcharacteristic variation of the pixel signal. The offset-compensatedpixel signal DS11 is then transferred to the processing block 16,whereby it is suitably compensated for a variation in its outputcharacteristic by the offset and gain compensations. The offset- andgain-compensated pixel signal DS12 from the block 16 is then transferredto a gain compensation block 152 on the side of processing block 17,whereby the signal is compensated by the gain compensation for itstemperature characteristic variation. Finally, the pixel signal DS13compensated for variations in output and temperature characteristics isobtained.

FIG. 18 shows a procedure of compensation operations by conducting theoffset compensation for a structure-derived variation in the outputcharacteristic of a pixel signal, the offset and gain compensations fora variation in its temperature characteristic and then the offset andgain compensations for a variation in its output characteristic.

In this instance, a pixel signal DS from the image sensor 7 is digitizedand transferred to an offset compensation block 111 on the side of theoutput characteristic variation compensation block 16, whereby the pixelsignal is subjected to the offset compensation for a variation in itsoutput characteristic to attain a correct level TH necessary forcompensating the variation in the temperature characteristic of thepixel signal. The offset-compensated pixel signal DS21 is transferred tothe processing block 17, whereby it is compensated for its temperaturecharacteristic variation by the offset and gain compensations. Thecompensated pixel signal DS22 from the block 17 is transferred to a gaincompensation block 112 on the side of processing block 16, whereby thesignal is subjected to the gain compensation for its outputcharacteristic variation. This process produces a pixel signal DS23compensated for the variations in both output and temperaturecharacteristics.

FIG. 20 shows another example of different output characteristics ofthree pixel signals A, B and C which may be resulted from thestructure-derived variations of corresponding light sensor circuits.

FIG. 19 illustrates the operation of the output compensating circuit 11.

A memory 10 holds an offset-compensation value OFS preset for obtaininga pixel output of H at a sensor current value Im. In an offsetcompensating portion 111, the offset compensation of each digitizedpixel signal DS is conducted by arithmetic operations (addition,subtraction) using the offset compensation value OFS. As the result ofthe offset compensation, three pixel signals (A, B and C) have the samecharacteristic in the logarithmic response region WB as shown in FIG.21.

In a gain compensating portion 112, the gain compensation for variationsin characteristics of three pixel signals in the non-logarithmicresponse region WA below the threshold value H is conducted byarithmetic operations (multiplication) by using correspondingmultipliers and the offset compensated signals DS1.

In practice, the offset-compensated pixel signal DS1 is checked whetherit is smaller than the threshold value H and, if so (i.e., it lies inthe non-logarithmic response region WA), it is further subjected to thegain compensation by the following arithmetic operations using aspecified multiplier MLT selected from the memory 10.Output←H−(H−Pixel Signal DS1)×MultiplierA result signal is output as digital output-compensated pixel signalDS2.

As the result of gain compensation, three pixel signals A, B and C havethe same characteristics in the non-logarithmic response region WA asshown in FIG. 22. In this instance, when the offset-compensated pixelsignal DS1 is larger than the threshold value (i.e., in the logarithmicresponse region WB) is directly output as a digital output-compensatedpixel signal DS2.

FIG. 24 shows another example of different output characteristics ofthree pixel signals TA, TB and TC at corresponding temperatures.

According to the present invention, the compensation of output signalsof the image sensor is normally conducted when the outputcharacteristics of respective pixel signals have substantially the samegradient in the logarithmic response region WB and different shapes inthe non-logarithmic response region WA.

FIG. 23 illustrates the operation of the output compensating circuit 15in the above-mentioned case.

The memory 14 holds an offset-compensation value T-OFS preset forobtaining a pixel output of TH at a sensor current value Itm. In anoffset compensating portion 151, the digitized pixel signals DS areoffset compensated by arithmetic operations (addition, subtraction)using the offset compensation values T-OFS. As the result of the offsetcompensation, three pixel signals TA, TB and TC at correspondingtemperatures have the same characteristics in the logarithmic responseregion WB as shown in FIG. 25.

In a gain compensating portion 152, the gain compensation for variationsin characteristics of three pixel signals in the non-logarithmicresponse region WA below the threshold value TH is conducted byarithmetic operations (multiplication) using corresponding multipliersand the offset compensated signal DS1′.

In practice, the offset-compensated pixel signal DS1′ is examinedwhether it is smaller than the threshold value TH and, if so, it isfurther processed by the gain compensation by the following arithmeticoperations using a specified multiplier T-MLT selected from the memory14.Output←TH−(TH−Pixel Signal DS1′)×MultiplierA result signal is output as an output-compensated digital pixel signalDS2′.

As the result of the gain compensation, three pixel signals TA, TB andTC with corresponding temperatures have the same characteristics in thenon-logarithmic response region WA as shown in FIG. 26. In thisinstance, when the offset-compensated pixel signal DS1′ is larger thanthe threshold value TH is directly outputted as an offset-compensateddigital pixel signal DS2′.

FIG. 27 shows a procedure for conducting the offset-compensation of apixel signal for a variation in its temperature characteristic, theoffset and gain compensation for a structure-derived variation in itsoutput characteristic and the offset and gain compensations for avariation in its temperature characteristic in the described order.

In this case, a pixel signal DS output from the image sensor 7 isdigitized and transferred to an offset compensation block 151 on theside of the temperature characteristic compensation processing block 17,whereby the pixel signal is compensated for a variation in itstemperature characteristic to attain a correct level H necessary forcompensating for a variation in the output characteristics of the pixelsignal. The offset-compensated pixel signal DS11 is transferred to theprocessing block 16, whereby the signal is adaptively compensated forits output characteristic variation by the offset and gaincompensations. The offset- and gain-compensated pixel signal DS12 fromthe processing block 16 is then transferred to a gain compensatingportion 152 on the side of processing block 17, whereby the signal issubjected to the gain compensation for its temperature characteristicvariation. Finally, a pixel signal DS13 compensated for both kinds ofcharacteristic variations is obtained.

FIG. 28 shows a procedure for conducting the offset compensation of apixel signal for a structure-derived variation in its outputcharacteristic, the offset and gain compensations for a variation in itstemperature characteristic and then the offset and gain compensationsfor variation in its output characteristic.

In this instance, a pixel signal DS from the image sensor 7 is digitizedand transferred to an offset compensating portion 111 on the side of theoutput characteristic compensating block 16, whereby the pixel signal issubjected to the offset compensation for a variation in its outputcharacteristic to attain a correct level TH necessary for compensatingfor a variation in the temperature characteristic of the same pixelsignal. The offset-compensated pixel signal DS21 is transferred to theprocessing block 17, whereby it is adaptively processed by conductingthe offset and gain compensations for a variation in its temperaturecharacteristic. The compensated pixel signal DS22 from the block 17 istransferred to a gain compensating portion 112 on the side of theprocessing block 16, whereby the signal is further subjected to the gaincompensation for a variation in its output characteristic. Finally, apixel signal DS23 compensated for both kinds of characteristicvariations is obtained.

With a high speed video camera according to the present invention, ahigh speed phenomenon is captured by using an image sensor capable ofcompensating for variations in output characteristics of respectivepixels (light sensor circuits), wherein data of sequential images(frames) taken by the image sensor from the phenomenon (fast varyingsubject) is stored temporally on a high speed memory and then read outfrom the memory to present the images on a display for analysis on thephenomenon.

In this instance, if the image data outputted in a time series by theimage sensor 7 is, as shown in FIG. 29, converted by an A-D converter 9into digital signals, compensated for variations in outputcharacteristics of respective pixels by a compensating device 18 andthen successively stored on a frame memory (or a buffer memory) 20 underthe control of a memory controller 19, it is necessary to perform thecompensation by the compensating device at a very high speed to followthe high speed phenomenon captured by the image sensor. The high speedvideo camera 7 according to the present invention uses a compensatingdevice 18 composed of components shown in FIG. 6 and 11 excepting animage sensor 7 and an A-D converter 9. In an embodiment according to thepresent invention, as shown in FIG. 30, image signals outputted from animage sensor 7 are stored directly (i.e., with no compensation) on aframe memory 20. Pixel signals once stored on the frame memory 20 aresuccessively read from the memory 20 under the control of a memorycontroller 19, compensated for variations of output characteristics ofrespective pixel circuits in a compensating device 18, and converted bya signal converter 21 into an image signal which is then displayed on adisplay device 22. In FIG. 30, numeral 23 designates a videorecording/reproducing control circuit which provides the memorycontroller 19 with a control signal for changing-over to storing animage signal on the memory or reading an image signal from the memoryaccording to an external switching command SC for changing over theoperating mode from recording to reproducing or the reverse.

In the above described construction, a sequence of images taken from thehigh speed phenomenon by the image sensor 7 can be stored on the framememory 20 in real time following the high-speed phenomenon. Whenreproducing the recorded image signals, the compensation of each imagesensor signal for output characteristic variations of respective pixels(sensor circuits) can be conducted at a normal processing speed (with noneed of following the high speed of taking images by the image sensor)by reading the image signal data from the frame memory 20 in accordancewith a frame rate of the display device 22. For example, a moment ofcollision of an automobile with an obstruction in the collisionexperiment was captured by the image sensor 7 by taking a sequence ofimages at a frame rate of 60 to 80 frames per millisecond and storingthe image data on the high speed frame memory 20. The stored images areread successively from the frame memory 20 at a frame speed (1/30 or1/60 sec.) adapted to the display device and presented on the displayscreen.

FIG. 31 shows a high speed video camera according to another embodimentof the present invention.

Images of a high speed phenomenon taken and recorded at a high speedthrough an image sensor 7 on the high speed memory cannot be monitoredon a display screen because the frame rate of the camera 7 (imagesensor) differs from that of a display device 22. Therefore, the shownembodiment is provided with a separate memory 24 for recording thereonthe image data from the image sensor at a frame rate adapted to thedisplay device under the control of the recording/reproducing controlcircuit 23. The stored image data is then read from the memory 24,compensated for variations in output characteristics of the sensorcircuits by a compensating device 18 and then provided to the displaydevice which can present successively images of the high speedphenomenon on its screen.

The memory 24 stores a frame image signal extracted at specified timing,which is then updated. This enables the output compensating device 18 tocompensate the sensor signal read from the memory 24 at a normalprocessing speed with no need to follow a high speed of capturing thephenomenon. If the frame rate of the image sensor 7 taking sequentialimages of a high speed phenomenon is not a multiple of a frame rate ofthe display device, then images (frames) are extracted with a skip attimings close to the frame rate of the display and stored on the memory24. This enables the display device 22 to simultaneously present asequence of images being taken by the image sensor 7, thereby allowingthe user to monitor in real time the state of visualizing the high speedphenomenon.

It is also possible for the frame memory 20 to have an area reserved forstoring image signals in accordance with the frame rate of the displaydevice, thereby omitting the need of providing the separate memory 24.

FIG. 32 shows another embodiment of the present invention, which isprovided with a display device 22 capable of selectively switching aframe rate from 1/30 seconds to 1/60 seconds and the reverse and alsoprovided with two corresponding memories 241 and 242 which areselectively used for storing image signals to be displayed at a framerate of 1/30 seconds and a frame rate of 1/60 seconds respectively andeach of which is selected by a memory selecting switch SW1 or SW2 of aswitching circuit 25 in accordance with a frame rate selecting signalprovided from the display device 22. Recording an image signal on aselected memory 241 or 242 is conducted under the control of arecording/reproducing control circuit 23 according to the frame rateselecting signal received from the display device 22.

INDUSTRIAL APPLICABILITY OF THE INVENTION

As is apparent from the foregoing, a high speed video camera accordingto an aspect of the present invention is capable of capturing any highspeed phenomenon by an image sensor using MOS type light sensor (pixel)circuits featured by a wide dynamic range with the least occurrence ofafterglow of pixels, recording data of images taken from the high speedphenomenon through the image sensor on a high speed memory, conductingcompensation of each image sensor signal read from the memory forvariations in output characteristics of respective pixel circuits basedon the read-out image data and displaying a sequence of images of thephenomenon on a display device according to the compensated image data.In other words, the high speed video camera can take video from the highspeed phenomenon by its image sensor and at the same time record thevideo data on its memory following the high speed phenomenon whileconducting compensation for variations in the image sensor outputs withease at a normal processing speed by reading the stored image signalsfrom the memory. This is a great advantage of the video camera forcorrectly visualizing high speed phenomena.

A high speed video camera according to another aspect of the presentinvention is capable of storing data of images of a high speedphenomenon taken by an image sensor on a separate memory at a speedadapted to a frame rate of a display device, compensating for variationsin outputs of the image sensor by reading the image data stored in theseparate memory, reproducing and displaying images of the phenomenonaccording to the compensated image data, thus enabling the user of thecamera to visually monitor the phenomenon in real time on the displaydevice.

1. A high speed video camera for taking video of high speed phenomena,comprising: an image sensor composed of a number of light sensorcircuits each representing a unit pixel and capable of producing in aphotoelectric converting element a sensor current proportional to aquantity of light falling thereon and converting the current into avoltage signal by a MOS type transistor with a logarithmic outputcharacteristic in a weak inverse state and including a means forremoving a charge accumulated in a parasitic capacitor of thephotoelectric converting element by changing a drain voltage of thetransistor to a value lower than a normal value for a specified periodto initialize each pixel before detecting a light signal; a memory forstoring therein image data output from the image sensor at a high speed;a compensating circuit for compensating for variations in respectivepixel signals from the image sensor by using compensation valuespredetermined for output characteristics of the respective pixels basedon the image data read from the memory; and a display device fordisplaying an image based on the compensated image data.
 2. A high speedvideo camera as defined in claim 1, characterized in that it is providedwith a memory for storing image data output from the image sensor byrecording thereon the image data at a speed corresponding to a framerate of the display device, the compensating device compensates forvariations in each pixel signal from the image sensor based on the imagedata read from the memory and means for displaying an image based on thecompensated image data on the display device.